Repository navigation
feat(flux2): run FLUX.2 attention on SageAttention when enabled - #9659
Merged
lstein merged 7 commits intoOct 16, 2026
Merged
Conversation
With `attention_backend: sage`, FLUX.2 denoising enters the SageAttention scope. RTX 4090, FLUX.2 Klein 9B FP8: 15.6 s → 15.1 s at 1024², 27.1 s → 23.1 s at 2048², peak VRAM unchanged; a blind review judged no image worse.
This was referenced Oct 3, 2026
Pfannkuchensack
marked this pull request as ready for review
October 12, 2026 20:44
lstein
added a commit
that referenced
this pull request
Oct 16, 2026
…9660) > [!NOTE] > Stacked PR 3 of 8: merge after #9659. It targets `main`, so until the PRs before it merge, its diff also shows their commits and the merges that carry them forward. This PR's own change is one commit, [`6c8718945a`](Pfannkuchensack@6c87189). ## Summary With `attention_backend: sage`, Z-Image denoising enters the SageAttention scope from #9657, so its eligible attention calls run on SageAttention. Z-Image Control passes an attention mask and therefore stays on PyTorch SDPA, like everything outside the loop. The default `auto` is unchanged. The docs page lists Z-Image under Supported models. ## Related Issues / Discussions Stacked on #9657 (SageAttention backend) and #9659, the PR before it. ## QA Instructions Z-Image FP8, model loaded, three arms per seed as described in #9657: | | 1024² | 2048² | | --- | --- | --- | | Step, default SDPA | 520 ms | 3184 ms | | Step, cuDNN-first SDPA | 490 ms | 2819 ms | | Step, SageAttention | 462 ms (−11 %) | 2231 ms (−30 %) | | Whole generation | 10.4 → 10.0 s (−4 %) | 33.7 → 25.9 s (−23 %) | | Peak VRAM in the denoise scope | unchanged | unchanged | | LPIPS against default SDPA, median (cuDNN-first) | 0.154 (0.092) | 0.168 (0.112) | - Z-Image reacts strongly to any kernel change: even the exact cuDNN swap moves LPIPS to 0.09–0.11, and SageAttention stays within 1.7× of that. Blind side-by-side review: 8 of 8 pairs equal; the pair with the largest LPIPS (0.25) shows the same scene, sharpness and detail. - Per step, the 32 calls of the main and noise-refiner layers run on SageAttention; the two text-only context-refiner layers are below the size threshold and stay on SDPA. - Accuracy on real activations, every call of step 0 (32 calls): worst cosine 0.9970, relative L2 0.043. - The timings, LPIPS and blind review come from the measurement prototype, which still used `sageattn()`'s default K smoothing; the selection #9657 ships is faster per call. To try it: install SageAttention as the docs page from #9657 describes, set `attention_backend: sage`, restart and generate with Z-Image. The first generation logs `SageAttention ... serves diffusion-model attention on cuda:0 (...)` with the GPU and kernel name, and with `log_level: debug` every generation logs how many calls ran on SageAttention and why the others did not. With `attention_backend: auto` the output is bit-identical to `main`. Checks: ruff clean; `pytest tests/app/invocations` passes with the whole stack applied (2090 passed). No call-site test was added: this change only enters the scope, whose behavior #9657 tests. ## Review No material findings for this call site. Z-Image Control gains nothing until its mask is handled (possible follow-up). ## Compatibility / Rollout Merge after #9659. No API or persisted-state change; only `attention_backend: sage` behaves differently. ## Checklist - [x] _The PR has a short but descriptive title, suitable for a changelog_ - [ ] _Meaningful regression coverage added / updated where needed; obsolete tests/code removed_ - [ ] _Persisted-state and API changes include required migrations / compatibility validation_ - [x] _Relevant performance/efficiency opportunities considered; material claims have evidence_ - [x] _Material review findings resolved and relevant checks rerun_ - [x] _Documentation added / updated (if applicable)_ - [ ] _Updated `What's New` copy (if doing a release after this PR)_
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Note
Stacked PR 2 of 8: merge after #9658. It targets
main, so until the PRs before it merge, its diff also shows their commits and the merges that carry them forward. This PR's own change is one commit,a1e97b7b09.Summary
With
attention_backend: sage, FLUX.2 denoising enters the SageAttention scope from #9657, so its eligible attention calls run on SageAttention. Everything else stays on PyTorch SDPA, and the defaultautois unchanged. The docs page lists FLUX.2 Klein 9B under Supported models.Related Issues / Discussions
Stacked on #9657 (SageAttention backend) and #9658, the PR before it.
QA Instructions
FLUX.2 Klein 9B FP8, model loaded, three arms per seed as described in #9657:
sageattn()'s default K smoothing; the selection feat(attention): add opt-in SageAttention backend for diffusion models #9657 ships is faster per call.To try it: install SageAttention as the docs page from #9657 describes, set
attention_backend: sage, restart and generate with FLUX.2. The first generation logsSageAttention ... serves diffusion-model attention on cuda:0 (...)with the GPU and kernel name, and withlog_level: debugevery generation logs how many calls ran on SageAttention and why the others did not. Withattention_backend: autothe output is bit-identical tomain.Checks: ruff clean;
pytest tests/app/invocationspasses with the whole stack applied (2090 passed). No call-site test was added: this change only enters the scope, whose behavior #9657 tests.Review
No material findings for this call site.
Compatibility / Rollout
Merge after #9658. No API or persisted-state change; only
attention_backend: sagebehaves differently.Checklist
What's Newcopy (if doing a release after this PR)